Incremental Reduced Support Vector Machines

The reduced support vector machine (RSVM) has been proposed to avoid the computational difficulties in generating a nonlinear support vector machine classifier for a massive dataset. RSVM selects a small random subset from the entire dataset with a user pre-specified size m̄ to generate a reduced kernel (rectangular) matrix. This reduced kernel will replace the fully dense square kernel matrix used in the nonlinear support vector machine formulation to cut the problem size and computational time and will not scarify the prediction accuracy. In this paper, we propose a new algorithm, Incremental Reduced Support Vector Machine (IRSVM). In contrast to purely random selection scheme used in RSVM, IRSVM begins with an extremely small reduced set and incrementally expands the reduced set according to an information criterion. This information-criterion based incremental selection can be achieved by solving a series of small least squares problems. In our approach, the size of reduced set will be determined automatically and dynamically but not pre-specified. The experimental tests on four publicly available datasets from the University of California (UC) Irvine repository show that IRSVM used a smaller reduced set than RSVM without scarifying classification accuracy.